Gesture-Free Spoken Dialogue with a Robotic Office Assistant

ion of m-pepsican, phys-obj-slot is activated. When m-pepsican is activated, the object models for pepsicans are brought into the robot's memory through a callback associated with m-pepsican unless the models are already in memory due to the previous activities. After m-phys-obj is activated, the token sequence of the MOP's expectation becomes empty, thus activating the MOP. When the MOP is activated, an instance MOP, call it m-get-physobj-12, is created under it with values m-get for the verb slot and m-pepsican for the object slot. The new MOP is placed under m-active-goal to reflect the fact that it is now one of the active goals pursued by the robot. As soon as m-get-phys-obj is activated, its callback installs (achieve m-get-phys-obj) on the RAP interpreter’s agenda. The RAP for achieving this goal checks for instances of the MOP m-get-phys-obj that are also instances of m-active-goal. In this case, the RAP finds m-get-phys-obj-12, extracts the m-pepsican value from the MOP's object slot and installs (get-phys-obj m-pepsican) on the RAP interpreter’s agenda. During execution, the RAP enables detect-obj-skill. The skill captures the image shown on the left in Figure 6, detects the pepsican (the right image in Figure 6 shows the detection region whitened), and puts the following assertions in the RAP memory: (detected-obj pepsican), (dist-to 1.5), (obj-xy 81 43), and (sim-score .73). These assertions state that the skill detected a pepsican 1.5 meters away from the robot, and the bottom left coordinates of the image region that had the best matching score of .73 are x=81, y=43. The get-phys-obj RAP invokes the homing skill. The skill causes the robot to navigate to the object. When the robot is at the object, the RAP enables the pickup skill so that the robot can pick up the can with the gripper. Finally, the RAP removes m-get-phys-obj-12 from under m-active-goal and places it under m-achieved-goal. Fig. 6. Command Execution. 4.2.2 Goal Disambiguation The system is designed to handle three types of goal ambiguity: sensory, mnemonic, and linguistic. Sensory ambiguity occurs when a goal referenced by the input is ambiguous with respect to sensory data. Mnemonic ambiguity takes place when the input references a goal ambiguous with respect to the robot's memory organization. Linguistic ambiguity arises when the linguistic input itself must be disambiguated to reference a goal. A goal is ambiguous with respect to sensory data when it refers to an object from a detected homogeneous set or when it refers to an abstraction common to all objects in a detected heterogeneous set. To understand sensory ambiguity, consider a situation when the robot has detected two pepsicans and the operator asks the robot to bring a pepsican. The homogeneous set of detected objects consists of two pepsicans. The input refers to a pepsican, i.e., an object from the detected homogeneous set. All things being equal, the robot